Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFC 0180] Remove broken packages or unmaintained leaf packages #180

Merged
merged 1 commit into from
Oct 28, 2024

Conversation

Mic92
Copy link
Member

@Mic92 Mic92 commented Jul 14, 2024

@Mic92 Mic92 force-pushed the removal-rfc branch 2 times, most recently from 3fd8a73 to 2e5572a Compare July 14, 2024 05:12
@Mic92 Mic92 changed the title [RFC 0178] Remove broken and unmaintained leaf packages [RFC 0180] Remove broken and unmaintained leaf packages Jul 14, 2024
@Mic92 Mic92 force-pushed the removal-rfc branch 3 times, most recently from 0082dff to 215d264 Compare July 14, 2024 05:25
Copy link
Member

@dotlambda dotlambda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There should be a rule to notify e.g. everyone who touched the package expression in the past before removal.

rfcs/0180-broken-package-removal.md Outdated Show resolved Hide resolved
rfcs/0180-broken-package-removal.md Outdated Show resolved Hide resolved
@Frontear
Copy link
Member

Frontear commented Jul 14, 2024

There are some packages with no maintainer yet don't exactly "become outdated" (an example is the intel-microcode package). How are we going to handle those?

@Mic92
Copy link
Member Author

Mic92 commented Jul 14, 2024

There are some packages with no maintainer yet don't exactly "become outdated" (an example is the intel-microcode package). How are we going to handle those?

I am sure that someone wants to become a maintainer of a package like this, once we actually become serious about removing it, because their hardware won't function without it. However we leave the decision what to do in this case up to the person merging the pull request i.e. waiting for a reasonable amount of time depending on the importance package.

rfcs/0180-broken-package-removal.md Outdated Show resolved Hide resolved
rfcs/0180-broken-package-removal.md Outdated Show resolved Hide resolved
rfcs/0180-broken-package-removal.md Outdated Show resolved Hide resolved
@AndersonTorres
Copy link
Member

There are some packages with no maintainer yet don't exactly "become outdated" (an example is the intel-microcode package). How are we going to handle those?

There are some of those packages that are very critical. Ideally they should be kept by a team.

@Mic92 Mic92 force-pushed the removal-rfc branch 2 times, most recently from 849ea46 to 086cb9a Compare July 14, 2024 06:02
@Mic92
Copy link
Member Author

Mic92 commented Jul 14, 2024

There are some packages with no maintainer yet don't exactly "become outdated" (an example is the intel-microcode package). How are we going to handle those?

There are some of those packages that are very critical. Ideally they should be kept by a team.

I think having this RFC in place, will make actually possible to find instances like this and hopefully improve the maintenance of these packages. I could see the security team for example maintaining microcode package.

@AndersonTorres
Copy link
Member

I could see the security team for example maintaining microcode package.

Or the nixos-hardware team (when they become a NixOS team :D )

@Aleksanaa
Copy link
Member

I hope the problem infrastructure or something like it can be implemented first, so we can add warnings to end users like "foo is deprecated and is subject to removal because ...", and guide them to give feedback. If they give reasonable reasons, we will cancel or postpone the removal.

We can divide the deprecation of a package into three stages:

  1. Throws a warning to end users but still builds on hydra
  2. Make a mark similar to insecure. If the user needs to use it, it needs to be manually allowed and it will not be built on hydra.
  3. Completely remove related expressions.

Each of these stages can last three to six months, giving everyone plenty of time to react and plenty of reason to remove it.

@Frontear
Copy link
Member

Additionally going back to maintainerless packages, I think itd be best to do a broad sweep and find which packages are still orphans and find them a team, especially for critical packages like the microcode package. Once this is done anything left behind would be non essential stuff, which can follow the removal procedure

@8573
Copy link

8573 commented Jul 14, 2024

Each of these stages can last three to six months, giving everyone plenty of time to react

Wouldn't anything shorter than six months per stage not be enough time for users of stable channels to notice before the removal process may move on to the next stage?

@AndersonTorres
Copy link
Member

About orphaning, we also need to deal with the silent retirement. There are a truckload of maintainers that did not maintain their codes from a long span of time.

I suggested to do a soft deletion of silent retired maintainers at NixOS/nixpkgs#310759.

Regardless the above:

We have the infamous Zero Hydra Failures event. We can do something similar around the months 3 and 9 in order to mark and sweep unmaintained packages.

@Mic92
Copy link
Member Author

Mic92 commented Jul 15, 2024

I hope the problem infrastructure or something like it can be implemented first, so we can add warnings to end users like "foo is deprecated and is subject to removal because ...", and guide them to give feedback. If they give reasonable reasons, we will cancel or postpone the removal.

We can divide the deprecation of a package into three stages:

1. Throws a warning to end users but still builds on hydra

2. Make a mark similar to insecure. If the user needs to use it, it needs to be manually allowed and it will not be built on hydra.

3. Completely remove related expressions.

Each of these stages can last three to six months, giving everyone plenty of time to react and plenty of reason to remove it.

I don't see really enough benefit in step 2, it just creates more work for us. Removing packages should not be more work than adding packages.

rfcs/0180-broken-package-removal.md Outdated Show resolved Hide resolved
rfcs/0180-broken-package-removal.md Outdated Show resolved Hide resolved
@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/rfcsc-meeting-2024-09-30/53690/1

@AndersonTorres
Copy link
Member

I believe situations like these worth some consideration:

NixOS/nixpkgs#345758

@Atemu
Copy link
Member

Atemu commented Oct 2, 2024

I fail to see how such situations have anything to do with this RFC.

@Naxdy
Copy link

Naxdy commented Oct 2, 2024

I think upstream disappearing is a special case, since all expressions involving the affected packages would no longer build without a binary cache providing the sources (even if nothing else changed), and there is no hope to restoring a functional state.

Since this RFC only mentions broken or unmaintained packages, and imo neither of those labels adequately describes the situation, I think we'd probably need a separate RFC for that one. There are also a lot of special cases to consider, like what if there's already a well-adopted fork out there, what in the case of legal takedowns, should these packages be removed only from unstable or also stable, etc. - things that are not immediately obvious at all. Probably a writeup for another day 😛

@Frontear
Copy link
Member

Frontear commented Oct 2, 2024

I actually agree with the above for another reason:

If a package disappears upstream, then it propagates to every single revision of nixpkgs that has ever provided that package.

To put it another way, were it not for the binary cache keeping packages, if upstream disappears that derivation essentially becomes one big build failure. Packages like that can potentially pose a problem for users who depended on said package, whether it be now or many releases ago. I do think there's some worth exploring the possible avenues of how to handle it, because there's a lot of nuance involved there, and a simple "mark as broken -> remove" may not be the best decision if we consider the impact radius.

EDIT: I realize I essentially re-stated what the above poster said 😝. I do think this concern has merit, and that this PR doesn't really apply much to it. It would be a good rfc for the future to consider how to deal with sources that have disappeared from the internet. One of nixpkgs' main benefits is being able to go back and use something from long before. Matter of fact I myself used this technique to compile a program which linked to glibc 2.34. There is a pretty decent concern for what should be done when this expectation breaks. Of course its out of nixpkgs' hands and I do not suggest we need to take responsibility in fixing said source, but I do think the discussion can be useful.

@infinisil
Copy link
Member

@emilazy @preisi @jopejoe1 @Mic92 You initiated FCP about 3 weeks ago, but there have been some discussions since then. Can you clarify the final decision on whether to accept/reject or have further discussion?

Copy link
Member

@AndersonTorres AndersonTorres left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reformat and extra suggestion

rfcs/0180-broken-package-removal.md Outdated Show resolved Hide resolved
rfcs/0180-broken-package-removal.md Outdated Show resolved Hide resolved
@Mic92 Mic92 force-pushed the removal-rfc branch 4 times, most recently from 6d66ee8 to 9bc51a0 Compare October 20, 2024 19:57
@Mic92
Copy link
Member Author

Mic92 commented Oct 21, 2024

@infinisil afaik all shepherds have agreed to this RFC. Now @jopejoe1 just needs to confirm.

@jopejoe1
Copy link
Member

@infinisil afaik all shepherds have agreed to this RFC. Now @jopejoe1 just needs to confirm.

Can confirm that all of us shepherds have agreed to this RFC.

Co-authored-by: Priyanshu Tripathi <[email protected]>
Co-authored-by: Anderson Torres <[email protected]>
Co-authored-by: Atemu <[email protected]>
Co-authored-by: Sefa Eyeoglu <[email protected]>
@kevincox
Copy link
Contributor

RFCSC: Thanks everyone, this RFC has been accepted!

@kevincox kevincox merged commit f557e07 into NixOS:master Oct 28, 2024
@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/rfcsc-meeting-2024-10-28/55095/1

@GetPsyched GetPsyched added status: accepted and removed status: FCP in Final Comment Period labels Oct 28, 2024
@Mic92 Mic92 deleted the removal-rfc branch October 29, 2024 13:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.